The Data Model is the interface allows pro users to prepare, condition, and structure data for analysis - effectively allowing them to create "Data Models" and ultimately "semantic models".
Most users generally do not build and create their own data models, instead consuming them as prebuilt items in Discover, Formulate and the other Pyramid tools - as such, the Model tools are usually for a select group of advanced data users or data scientists.
What is Data Modeling
Data modeling allows users to create a proper data fabric architecture for reporting, analytics and decision making. It involves either the ingestion of data to one or more target technologies or the direct querying of these entities. It almost always involves adding a virtual semantic model, so the querying tools know how to connect the data elements together. However, it may also involve some very complex flows and pipeline to fix data, orchestrate events and ultimately trigger internal or external applications.
- Click here to see the Data Models tutorial video library
Data Flows vs Decision Flows
Decision and Data flows are part of the same "Model" app in Pyramid. However, data flows are earmarked and optimized for data modeling, data preparation and DSML activities and therefore must include at least one data preparation process. Decision flows, on the other hand, do not require a data flow sequence and can run with the master flow tools agnostically.
Key Data Modeling Concepts
The Model tool offers powerful tools to end users for preparing and querying data and there are several key concepts that will better explain how this toolset works in Pyramid.
- External Models vs Pyramid Models: How to mix and match data models built in Pyramid vs those brought in from 3rd party engines like MS OLAP and SAP BW.
- Model Structures: Key elements of the modeling workflow in Pyramid include understanding the difference between the model definition files vs the materialized databases and semantic models.
- Data Flows vs Semantic Models: This explains the 2 core frameworks in the Model app - flows for manipulating and blending data, and modeling for creating the virtual semantic data model layers.
- Click here for more details on the key concepts above.
Model Tool Interfaces
Pyramid offers four ways (aka "modalities") to access Model, with varying degrees of functionality and sophistication. They can be broken out into "quick" and "full" options.
Quick Modeling
Use the quick modeling tools if you want the application to automatically built the data model for you:
- Smart Model: import a file as a datasource, and the augmented AI modeling engine will build the data model for you.
- Direct Model: this option allows you to quickly add a new virtual semantic data model to an existing database without any data manipulation options.
Full Modeling
Use the full modeling tools to build the necessary steps to perform data manipulation, add data logic, and build data models:
- Model Lite: this option provides users with a simple, step-by-step wizard for blending data sets and building data models.
- Model Pro: with this option, proficient users can build custom data flows ("ETL"), perform advanced data manipulation (like machine learning, scripting engines, column operations, and more), design and build data models, set security and orchestrate automated refreshes and updates.